SMART in TREC 8

This year was a light year for the Smart Information Retrieval Project at SabIR Research and Cornell. We oÆcially participated in only the Ad-hoc Task and the Query Track. In the Ad-hoc Task, we made minor modi cations to our document weighting schemes to emphasize high-precision searches on shorter queries. This proved only mildly successful; the top relevant document was retrieved higher, but the rest of the retrieval tended to be hurt. Our Query Track runs are described here, but the much more interesting analysis of these runs is described in the Query Track Overview. Basic Indexing and Retrieval In the Smart system, the vector-processing model of retrieval is used to transform both the available information requests as well as the stored documents into vectors of the form: Di = (wi1; wi2; : : : ; wit) where Di represents a document (or query) text and wik is the weight of term Tk in document Di. A weight of zero is used for terms that are absent from a particular document, and positive weights characterize terms actually assigned. The assumption is that t terms in all are available for the representation of the information. The basic \tf*idf" weighting schemes used within SMART have been discussed many times. For TREC 8 we made a slight modi cation to Lnu-ltu weights we have used in the past 4 years in TREC 4{7. We noticed that the pivoted byte-length document normalization used by Singhal et al in TREC 7([5]) seems to favor high precision searches when used with short queries. It is a bit more biased towards shorter documents than our previous \u" scheme which uses number of unique terms in the document, thus a good short document containing all the query terms will be ranked highly. We hoped that this would enable our blind feedback query expansion to be based on more relevant documents and thus improved. The same phrase strategy (and phrases) used in all previous TRECs (for example [2, 3, 4, 1]) are used for TREC 8. Any pair of adjacent non-stopwords is regarded as a potential phrase. The nal list of phrases is composed of those pairs of words occurring in 25 or more documents of the initial TREC 1 document set. Phrases are weighted with the same scheme as single terms. Note that no human expertise in the subject matter is required for either the initial collection creation, or the actual query formulation. When the text of document Di is represented by a vector of the form (di1; di2; : : : ; dit) and query Qj by the vector (qj1; qj2; : : : ; qjt), a similarity (S) computation between the two items can conveniently be obtained as the inner product between corresponding weighted term vectors as follows: